🕷️️ Job Radar • SCRAPING

freelancer.com 🟡 2026-05-11

🔹 Real-Time Pricing Data Scraper
👤 Client: 🇬🇧 Billericay, United Kingdom Member since 2026-04-24
💰 Price: $8 / hr Average bid
🚩 Problem: Automate continuous polling of six websites for real-time pricing data updates.
📦 Existing: [Current Tech: Chrome-based internal tooling]

Specifications:

[Target] Real-time price updates every few seconds across multiple sites.
[Method] Continuous scraping or tight scheduling with robust error handling and logging.
[UI/UX] Not applicable (headless solution).
[Stack] Node.js, Puppeteer, Kafka for message queue, Express for RESTful API.
[Security] Secure data transmission and storage. Implement rate limiting to avoid detection.
[Format] JSON payloads pushed to RESTful endpoint or message queue.

Workflow:

1. Analyze current internal tooling and identify key features to maintain.
2. Develop a headless scraping solution using Puppeteer for each target website, ensuring compatibility with specified data points.
3. Implement real-time polling logic with configurable intervals (e.g., every few seconds).
4. Set up robust error handling and logging mechanisms to ensure continuous operation even under site changes or rate limits.
5. Integrate scraping results into a message queue (Kafka) for immediate processing or push directly to RESTful endpoint via Express server.
6. Test the solution thoroughly with all six target websites, ensuring data accuracy and system stability.